Enhancing image data for different types of displays

ABSTRACT

A method is provided for generating enhanced image data for display on different types of display platforms. The method can include receiving a first format of display image data mastered to be displayed on a first type of display system. The method can include receiving a second format of display image data mastered to be displayed on a second type of display system that is different than the first type of display system. The method can include detecting one or more corresponding pixels in image segments among the first format of display image data and the second format of display image data. The method can include using attributes from the one or more corresponding pixel to generate enhanced image data for display.

CROSS REFERENCE TO RELATED APPLICATION

This claims priority to U.S. Provisional Application Ser. No.63/086,668, filed Oct. 2, 2020 and titled “ENHANCING IMAGE DATA FORDIFFERENT TYPES OF DISPLAYS,” and to U.S. Provisional Application Ser.No. 63/090,078, filed Oct. 9, 2020 and titled “ENHANCING IMAGE DATA FORDIFFERENT TYPES OF DISPLAYS,” the entirety of each of which isincorporated herein by reference.

FIELD OF THE DISCLOSURE

This disclosure relates generally to digitally enhancing content ofimage data such as motion picture data and, more specifically (althoughnot necessarily exclusively), to modifying the image content to bepresented on different types of display platforms.

BACKGROUND

Image content, such as a motion picture or a video production, can bemastered for a designated display platform in a professional facility.Mastered image content may or may not include a content creatorapproving the visual appearance of the image content based on the visualimagery displayed on a display device representative of the displayplatform. For example, mastering a motion picture may be performed in apost-production facility equipped with a DCI-certified cinema projector.Similarly, a video production intended for broadcasting may be masteredin a production suite equipped with calibrated professional monitors.With an increasing popularity of on-demand streaming services availableto homes and mobile devices, a motion picture or a video production maybe released to such display platforms in addition to a wide range ofother display platforms, including home theatres, indoor signage panels,and outdoor billboards. The different display platforms have verydifferent characteristics among each other, including aspect ratio,resolution, frame rate, color gamut, brightness level, contrast, dynamicrange, black level, and white point. The image content may appear vastlydifferent on each display platform unless professionally mastered foreach display device. Moreover, within each display platform, traditionaldisplay technologies are being rapidly replaced by newer generations ofdisplay technologies such as LED, QD, OLED, MicroLED, and laserprojection, and each new generation of display technology inevitablyintroduces a distinct and novel visual appearance.

For example, peak brightness is one of the characteristics that can beused to characterize a display platform. Peak brightness is the maximumlight output capacity of the display platform. A standard cinema isdesigned for a dark viewing environment, and it can provide a peakbrightness of 48 nits on a projection screen. A typical IMAX® theatrescreen can provide a nearly 60% higher peak brightness level for alarge-format cinema viewing environment. A direct-view LED cinema screencan deliver 300 nits of light full screen, while a projector based onlight-steering technology can provide even higher brightness level inhighlight regions. Home TV and mobile devices can provide higher peakbrightness levels because those devices are designed for a much brighterviewing environment. Certain brands of OLED TV screens can exceed 600nits of peak brightness, while other types of TV systems may exceed 1000nits. Some mobile phone screens are designed for daylight viewingconditions and can produce even higher brightness levels. Differentdisplay platforms may also differ greatly with other characteristicssuch as color, contrast, and dynamic range. When a motion picture or avideo production is to be released to such a wide range of displayplatforms, it becomes increasingly difficult and cost prohibitive tomaster the image content for potentially applicable display platformsand display technologies.

The appearance of image content can be enhanced based on characteristicsof a display device by using image processing techniques. One examplecan be based on an appearance graph. An image sequence is processedusing various methods to render multiple prime layer sequences, eachwith a distinctive visual appearance. The prime layer sequences arecombined to produce a new image sequence with a new visual appearancebased on an appearance graph. The visual appearance of the resultingimage sequence can be further fine-tuned by adjusting the contributionsfrom prime layer sequences. Certain prime layer sequences may becomputed based on pixel motion information including optical flow ormore accurate trajectories. Such an appearance-graph based methodprovides only a limited range of visual appearance enhancement and,therefore, may not be adapted to a wide range of display platforms andviewing conditions without involving costly mastering treatments thatare time consuming and impractical.

SUMMARY

In one example, a method includes receiving a first format of displayimage data mastered to be displayed on a first type of display system.The method includes receiving a second format of display image datamastered to be displayed on a second type of display system that isdifferent than the first type of display system. The method includesdetecting one or more corresponding pixels in image segments among thefirst format of display image data and the second format of displayimage data. The method also includes using attributes from the one ormore corresponding pixels to generate enhanced image data for display.

In another example, a method of image enhancement includes receiving afirst format of display image data mastered to be displayed on a firsttype of display system. The method includes receiving a second format ofdisplay image data mastered to be displayed on a second type of displaysystem that is different than the first type of display system. Themethod includes generating segmented image data by comparing the firstformat of display image data and the second format of display image datato determine a plurality of image data similarities and a plurality ofimage data differences. The method includes determining, based onspatial correspondence or temporal correspondence, a corresponding pixelpair between the first format of display image data and the secondformat of display image data from the segmented image data. The methodincludes defining a local mask sequence using the corresponding pixelpair. The method also includes outputting the local mask sequence as amask file for generating enhanced image data to be displayed.

In another example, a system comprises a processing device and anon-transitory memory comprising instructions that are executable by theprocessing device for causing the processing device to performoperations. The operations include receiving a first format of displayimage data mastered to be displayed on a first type of display system.The operations include receiving a second format of display image datamastered to be displayed on a second type of display system that isdifferent than the first type of display system. The operations includedetecting one or more corresponding pixels in image segments among thefirst format of display image data and the second format of displayimage data. The operations also include using attributes from the one ormore corresponding pixels to generate enhanced image data for display.

In another example, an image enhancement system comprises a processingdevice and a non-transitory computer-readable memory comprisinginstructions that are executable by the processing device for causingthe processing device to perform operations. The operations includereceiving a first format of display image data mastered to be displayedon a first type of display system. The operations include receiving asecond format of display image data mastered to be displayed on a secondtype of display system that is different than the first type of displaysystem. The operations include generating segmented image data bycomparing the first format of display image data and the second formatof display image data to determine a plurality of image datasimilarities and a plurality of image data differences. The operationsinclude determining, based on spatial correspondence or temporalcorrespondence, a corresponding pixel pair between the first format ofdisplay image data and the second format of display image data from thesegmented image data. The operations include defining a local masksequence using the corresponding pixel pair. The operations also includeoutputting the local mask sequence as a mask file for generatingenhanced image data to be displayed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic of an environment for modifying image content fordifferent display devices according to one example of the presentdisclosure.

FIG. 2 is a flow diagram of a process for enhancing image content to bedisplayed on different types of displays according to one example of thepresent disclosure.

FIG. 3 is a flow chart of a learning-based process implemented by acontent comparison processor according to one example of the presentdisclosure.

FIG. 4 is a flow chart of a process for producing enhanced image data bya display content generator according to one example of the presentdisclosure.

FIG. 5 is a block diagram of a system for enhancing image data accordingto one example of the present disclosure.

FIG. 6 is a block diagram of a configuration for enhancing image datafor display on a different type of display device according to oneexample of the present disclosure.

FIG. 7 is a block diagram of a second configuration for enhancing imagedata for display on a different type of display device according to oneexample of the present disclosure.

DETAILED DESCRIPTION

Certain aspects and features relate to enhancing one or morecharacteristics of a sequence of image data by analyzing attributesbetween at least two versions of the sequence of image data, eachmastered for a specific display device and a specific viewingenvironment. The image analysis can be performed globally across theimage content and locally at a specific part of the image content. Anenhanced sequence of image data can be generated by transferringselected attributes between two versions of the sequences of image data.The enhanced version may be an enhancement of one or both versions ofthe sequence of image data. In other examples, the enhanced version maybe in a third version of the sequences of image data that is enhancedimage data for display on a different type of display device than thedisplay devices on which the two versions were mastered for display.

A sequence of image data, or an image sequence, includes multiple framesof images with motion information. Each image frame includes multipleimage pixels. An image pixel can define a picture element at aspace-time location that is related to neighboring image pixels in thesame image frame and may also be related to similar pixels across otherimage frames in the same image sequence. Each image pixel is a sample ofan original image at a specific space-time location, and an image pixelis typically represented by a set of triples or quadruples of values,such as RGB, XYZ, or CMYK values. The pixel values define the content ofan image or an image sequence, but the actual appearance of the imagesalso depends on the characteristics of a display platform where theimages are to be displayed.

A display platform can include a display hardware device, software andfirmware for display functions, and a viewing environment. Theappearance of image content displayed from a display platform can bedepend on certain characteristics of the display platform. Examples ofsuch characteristics include aspect ratio, resolution, frame rate, colorgamut, brightness level, contrast, dynamic range, black level, and whitepoint. In one example, a standard dynamic range (SDR) display platformmay be limited to a peak brightness of 200-300 nits, while a highdynamic range (HDR) display platform may support a peak brightness of1000 nits or higher. An HDR display platform can also provide a darkerblack level and increase overall contrast of the display platform. Inanother example, a standard digital cinema projector with full DCI-P3support can reproduce a color range of approximately 45% of visiblecolors, while a wide color gamut (WCG) display as recommended by ITU-RBT.2020 may cover a wider color range—up to over 75% of visible colors.In addition, the software and firmware implemented in a display platformmay also alter the appearance of image content. As a result, imagecontent, such as a motion picture or a streaming video, may appear verydifferently on various types of display platforms. If the image data ofa motion picture previously mastered for an IMAX® cinema screen isreleased directly to a direct view LED cinema screen (which may bebrighter and have an extended dynamic range) without additionalmastering, the images can appear flat, the color can be washed out, andimage quality can be significantly compromised.

A mastering process for motion pictures or video content can establishthe appearance of image content for presentation on a designated displayplatform in an approved viewing environment, and the final appearance ofimage content can be approved by the content creator at a masteringfacility. The image data of the approved image content is considered tobe mastered for the designated display. A typical mastering process mayinclude color grading in which various attributes of an image contentare modified, including color, brightness, contrast, dynamic range,detail, black level, white point, etc. Mastering can be performed in aprofessional facility equipped with appropriate display devicescalibrated for a certain viewing environment. The display device and theviewing environment is representative of the designated display platformto which the image content is to be released. For example, a motionpicture can be mastered for a DCI-complaint cinema environment, and thefacility can be equipped with a DCI-certified digital cinema projector,a projection screen with the correct screen gain, and a dark viewingenvironment that represents the environment of a typical movie theatre.Color grading can be applied to each scene of the motion picture so thatimage color appears correct and consistent. A mastering process may alsoinclude HDR up-conversion or tone mapping when the dynamic range of theimage content is to be extended or compressed. An HDR up-conversionprocess can extend the dynamic range of SDR source image content for acertain HDR displays, and it can be applied through an automated processor an interactive process. A tone-mapping process can compress thedynamic range of source image content to match lower dynamic rangedisplays, and the process can be automated. A mastering process may alsoinvolve modifying the resolution or aspect ratio of the image content toconform to the resolution and the aspect ratio of the intended displayplatform.

A professional mastering facility can be calibrated for a viewingenvironment that represents normal viewing conditions for a designateddisplay platform. A motion picture is typically mastered in a darkenvironment representative of a cinema where the amount of ambient lightis minimized. A video production suite may allow a certain amount ofambient light in simulation of a home viewing environment. Significantchanges to the viewing environment may result in unacceptable imageappearance distortion unless properly compensated. For example, when themotion picture is to be released for home TV or for mobile devices witha much brighter viewing environment than a theatre, additional colorgrading can be applied to compensate appearance distortions from theexistence of ambient light or daylight. In addition, due to thetime-varying nature of motion pictures or video content, certainmastering methods may track the motion information in the image content,including by using algorithms for estimating pixel motion and trackingpixel movement over multiple image frames. Motion estimation andtracking may include optical flow-based methods and more sophisticatedtrajectory-based methods that can be complex and computationallyexpensive. Mastering image content for each applicable display platformand for a broad range of viewing conditions can become time consuming,impractical, and cost prohibitive.

In some examples, image content can be modified so that the imagecontent may be displayed on different display platforms with aconsistent and approved appearance without involving costly masteringprocesses. Image content can be processed using a sequence of referenceimage data that is characterized with selected attributes such that theimage content can be enhanced by transferring some of the attributesfrom the reference image data to the image content. Image content canalso be processed for an unknown display platform using reference imagedata that is characterized with selected attributes such that the imagecontent is enhanced by transferring some of the attributes from thereference image data to the image content for the unknown display toachieve a preferred appearance.

These illustrative examples are given to introduce the reader to thegeneral subject matter discussed here and are not intended to limit thescope of the disclosed concepts. The following sections describe variousadditional features and examples with reference to the drawings in whichlike numerals indicate like elements but, like the illustrativeexamples, should not be used to limit the present disclosure.

FIG. 1 is a schematic of an environment for modifying image content fordifferent display devices according to one example of the presentdisclosure. Source image content 10 can be mastered for a first displayplatform 22 through a mastering process module 40, resulting in a firstversion of image data that is first format of display image data 20. Thefirst format of display image data 20 can provide a preferred appearancewhen displayed on the first display platform 22. The same source imagecontent 10 can also mastered through a mastering process module 40 for asecond display platform 32 with a different set of characteristics,resulting in a second version of image data that is a second format ofdisplay image data 30. The second format of display image data 30 canproduce a preferred appearance when displayed on the second displayplatform 32. Because the first format of display image data 20 and thesecond format of display image data 30 are mastered for differentdisplay platforms, each may acquire different attributes, such asdifferences in color, brightness, contrast, dynamic range, detail, blacklevel, white point, aspect ratio, resolution, frame rate, and dataformat, even though the versions originate from the same source imagedata. In some cases, the first format of display image data 20 and thesecond format of display image data 30 may result in different imageframes as the result of individual editing processes.

A processing method module 50 can convert the first format of displayimage data 20 to a modified version of image data that is an enhancedimage data for display 70, while using the second format of displayimage data 30 as a reference image data. The enhanced image data fordisplay 70 can be produced such that certain types of attributes aretransferred from the second format of display image data 30 into thefirst format of display image data 20, and the enhanced image data fordisplay 70 can provide a preferred appearance when displayed on a thirddisplay platform 72 that has different characteristics from the firstdisplay platform 22. The third display platform 72 may or may not sharesimilar characteristics with the second display platform 32. In the caseof the third display platform 72 being vastly different then the seconddisplay platform 32, the processing method module 50 may receive certaininformation that defines the characteristics of the third displayplatform 72. Such information may also include the descriptions of theintended viewing conditions for the third display platform 72. Thecharacteristics of the third display platform 72, and the descriptionsof the intended viewing conditions for the third display platform 72,may be included in third display specifications 60. The informationincluded in the third display specifications 60 may be received from themanufacturers of the third display platform 72, or it can also bereceived from measurements of the third display platform 72 undercertain viewing conditions.

In one implementation, the source image content 10 may be a motionpicture, and the first display platform 22 may be a digital cinema thatmeets DCI specifications including a peak brightness of 14 foot lambertsor 48 nits and a DCI-P3 color space. The first format of display imagedata 20 may be a digital cinema package (DCP) of the motion picture thatis mastered for and distributed to the digital cinema that meets DCIspecifications. The second display platform 32 may be a HDR-capable TVdisplay that supports a peak brightness up to 1000 nits, a much lowerblack level and Rec 2020 color gamut. Examples of the second displayplatform 32 may be certain brands of OLED and quantum dots home TVdevices. The second format of display image data 30 may be a HDR versionof the motion picture mastered for distribution by a streaming servicesuch as Netflix™ HBO™, Hulu™, or Disney+™. The second format of displayimage data 30 may have a streaming data format with advanced videocoding. The third display platform 72 may be a direct-view LED cinemascreen that supports a peak brightness level of up to 300 nits, with anextended dynamic range and a wider color gamut, in comparison with thefirst display platform 22. An existing motion picture DCP can beconverted to an enhanced DCP for release to an LED cinema, and theenhanced image data for display 70 can be the enhanced DCP.

In another example, the first format of display image data 20 may be anHDR version of a motion picture, formatted for HDR streaming services,for a first display platform 22 that is an HDR-capable TV device. Thesecond format of display image data 30 may be a DCP of the same motionpicture specially mastered for IMAX™ theatre releases for a seconddisplay platform 32 that is an IMAX™ theatre with a higher set oftechnical specifications than a conventional digital cinema, including ahigher peak brightness, a higher picture resolution, and a differentlarge format cinema environment. The IMAX™ DCP can be a cleaner andsharper version of the motion picture that is specially mastered usingan image enhancement computing process such as IMAX™ DMR or IMAX™digital mastering process, in which additional image detail may berecovered and unwanted noise suppressed. The HDR version of the motionpicture can be enhanced with the cleaner and sharper version of theIMAX™ DCP. The result can be enhanced image data for display 70, whichmay be a higher-quality HDR streaming release of the motion picture foradvanced services including IMAX™ enhanced services.

In another example, the first format of display image data 20 may be anIMAX™ DCP of a motion picture, and the second format of display imagedata 30 may be an HDR version of the motion picture mastered for homeTV. The third display platform 72 may be a home theatre equipped with a150-inch, direct-view LED screen. The LED screen may be based onadvanced mini-LED or MicroLED modular panels that can deliver a peakbrightness level of up to 1000 nits and a much higher contrast ratio.The LED home theatre can be HDR capable and can operate under a home,room-type environment with ambient lighting. In this case, the movietitles from an IMAX™ film library can be enhanced for LED home theatre.The enhanced image data for display 70 may be an IMAX™ home HDRdistribution package.

FIG. 2 is a flow chart of a process for enhancing image content to bedisplayed on different types of displays according to one example of thepresent disclosure. First format of display image data 20 can beprocessed with certain types of attributes that are enhanced to delivera preferred appearance on a third display platform. Second format ofdisplay image data 30 can be selected for demonstrating at least aportion of such attributes, such as a higher dynamic range, or a widercolor gamut, or a higher picture resolution. In some examples, thesecond format of display image data 30 serves as a reference image datafor including a number of preferred attributes. A learning-based processcan be introduced to enrich the first format of display image data 20with certain preferred attributes from the second format of displayimage data 30. The resulting enhanced version of the image data is theenhanced image data for display 70 that can exhibit a preferredappearance when displayed on a third display platform 72 in which thecharacteristics can be described by third display specifications 60.

The learning-based process in FIG. 2 can include a content comparisonprocessor 210, a metadata generator 220, and a display content generator230. In some examples, these are software modules that can be executedby a computing device to perform certain operations. For example, thecontent comparison processor 210 can compute and identify preferredattributes from the second format of display image data 30 for use toenhance the first format of display image data 20. The computing outputdata 212 from the content comparison processor 210 can be packaged in acompact form of metadata 222 by the metadata generator 120 to facilitateefficient and secure distribution. The metadata 222 may be used by thedisplay content generator 230 to render the enhanced image data fordisplay 70, which can further be fine-tuned based on the informationfrom the third display specifications 60 to achieve a desired appearanceon a third display platform.

FIG. 3 is a flow chart of a learning-based process that can beimplemented by a content comparison processor according to one exampleof the present disclosure. For example, the content comparison processor210 of FIG. 2 can receive first input image data as the first format ofdisplay image data 20, and can receive second input image data as thesecond format of display image data 30. The two versions of input imagedata are mastered for different display platforms, and each may haveacquired different attributes as a result of different masteringprocesses. For example, the first format of display image data 20 may bein a format of a cinema DCP with a 4K resolution, 1.896:1 aspect ratio,and a 24 fps frame rate, mastered for a cinema display with a peakbrightness of 75 nits and DCI P3 color space. And, the second format ofdisplay image data 30 may be in a form of a streaming video with a HDresolution, a Cinemascope aspect ratio of 2.39:1, and a 60 fps framerate, mastered for a HDR display with a peak brightness of 1000 nits andRec 2020 color space. In this example, the two versions of input imagedata can have different attributes, such as different frame count for ascene, different pixel count in a frame, or different pixel values for asame content.

In block 306, the input image data are compared to detect correspondingframes between the first format of display image data 20 and the secondformat of display image data 30. The corresponding frames between thetwo versions of input image data can form a collection of the imageframes from both versions that represent similar image content but mayhave different attributes. Because the two versions of input image datamay have different frame rates or may have different edits done, thecorresponding frames from the two versions may differ in frame counts orin length duration. Detecting corresponding frames may be based on theanalysis of certain global attributes that are shared by both versionsof the input image data.

A global attribute can be an attribute that is present over most or allof an image segment. Picture resolution is an example of a globalattribute since picture resolution is typically applicable over theentirety of an image segment. Another example is color range, whichincludes the same color grading applied to the entire image segment atthe mastering stage. Other examples of global attributes may includedynamic range, luminance distributions, and sharpness. These examplesmay jointly determine the appearance of the input image data. Anotherexample of a global attribute can include motion statistics.

In contrast to global attributes, local attributes are attributes thatare applicable to local areas of an image frame or of an image segment,rather than to most or all of an image segment. Examples of localattributes can include picture details within occluded regions, picturedetails within certain highlight areas, or noise in certain darkregions, etc. Local attributes also include attributes that cannot bedefined by global features or attributes that are properly transferredby more complex local operations, such as certain color saturation thatmay not be mathematically modeled within a color system.

In block 308, image segments are computed by breaking down thecorresponding frames into image segments using image analysiscomputations. An image segment can represent a continuity of an actionat a specific scene. The frame count of an image segment may varybetween two versions of the input image data because the versions mayhave different frame rates or have different edits. As an example, a10-second long image segment from the first format of display image data20 can contain 240 frames, but the same image segment from the secondformat of display image data 30 may have 600 frames. Computing imagesegments may be based on the analysis of changes of certain globalattributes in the temporal domain, such as changes of color, luminance,and motion statistics. The analysis of continuity of certain globalattributes may be used to refine the segmentation decisions. As aresult, each image segment from block 308 can include the image framesfrom both versions of the input image data, and the two versions ofimage data represents similar image content but may contain differentattributes including different frame counts.

Image segments with image frames from both versions of the input imagedata can reflect global attributes and local attributes. The imagesegments can be further analyzed to identify the types of attributesthat may be transferred from the second input image data to enhance thefirst input image data. The different types of attributes—globalattributes and local attributes can be further processed separately anddifferently. Global attributes can be processed via blocks 312, 314, and316. Local attributes can be processed via blocks 322 and 324.

In general, transferring a global attribute from the second input imagedata to the first input image data may be performed by computing aglobal feature transform. A global attribute may be defined as a globalfeature with a mathematical description. For example, the dynamic rangeof an image segment can be mathematically modeled by a pixel luminancedistribution curve. A mathematical transform that converts one versionof the global attribute to another version can be referred to as aglobal feature transform. Global feature transforms may include, but arenot limited to, scalers, matrixes, linear or nonlinear transferfunctions, affine transforms, look-up tables including 3D LUTS, andconversion algorithms. For example, the conversion of color ranges fromone tristimulus color system to another tristimulus color system can berealized through a matrix transformation. In some cases, a globalfeature transform may be complex such that it is described by analgorithm solution. For example, picture resolution can be a globalattribute, and the conversion of an image to a higher resolution imageinvolves an algorithm solution such as an upscaling algorithm or a moresophisticated super-resolution algorithm.

Blocks 312, 314 and 316 can compute the global feature transforms 330for the transfer of global attributes between versions of input imagedata. In block 312, each image segment is analyzed to identify pixelsthat are available in both versions of input image data. A portion ofthe pixels in one version of input image data may not exist in the otherversion, due to, for example, differences in aspect ratio or framecropping. But, the pixels that are jointly shared by both versions ofinput image data can form a related pixel group, and related pixelgroups for the image frames in each image segment can be identified. Arelated pixel group may be identified based on the types of globalattributes among the different versions of input image data. In block314, each related pixel group is analyzed to determine global features.Subsequent to determining global features, a global feature transform ora set of global feature transforms 330 are computed in block 316. Theglobal feature transforms 330 can be used to transfer certain globalattributes from the second input image data to the first input imagedata by a display content generator.

Local attributes from the image segments can be processed separatelyfrom global attributes, such as by being processed in blocks 322 and324, which can identify the local attributes that are to be transferredfrom the second input image data and can determine how local operationscan be applied for the transfer of the attributes to the first inputimage data. In block 322, each image segment is further analyzed todetermine if there is additional information from the second input imagedata that is missing from the first input image data. Examples ofadditional information may include extra image areas, higher resolutiondetails in certain areas, revealed details in occluded areas due toincreased frame rate, new details discovered in certain dark areas ofthe image frames, new details revealed in some highlight areas due toincreased dynamic range, saturated colors outside the current colorgamut limitation, etc. A corresponding pixel pair is an image pixel inan image segment of a first input image data that has a correspondingimage pixel in an image segment of a second input image data.Corresponding pixel pairs may be identified based on analyzing spatialcorrespondence or temporal correspondence between the first input imagedata and the second input image data. Corresponding pixel pairs can markand reveal image areas with additional information from both versions ofinput image data.

In block 324, each corresponding pixel pair is further analyzed todetermine what types of local transfer operation is needed forenhancement. One or more sets of local mask sequences 332, 334, 336 canbe generated to provide guidance for each type of local transferoperations. A local mask sequence can be a form of a sequence ofgrayscale images in which the locations where local transfer operationsmay be needed in a particular frame are marked. In addition, a localmask sequence may also provide parameters to guide the local transferoperation to be applied correctly. In one example, the local transferoperation may include an image sharpening function, and the local masksequence may include sharpening level values for each pixel location. Inaddition or alternatively, the local transfer operation may include ablending operation, and the local mask sequence may provide blendingopacity values. A local mask sequence can be outputted as a mask fileand can be used to transfer certain local attributes between the firstimage content data and the second image content data.

In FIG. 3 , three types of local mask sequences are listed, includinglocal dark area masks 332, local highlight area masks 334, and localhigh saturation area masks 336. The local dark area masks 332 may begenerated for detail recovery in certain dark areas of the image. Suchdetails may be crushed or significantly suppressed in the first inputimage data but may be preserved in the second input image data as aresult of an extended dynamic range or more careful post-productionpractices. The local highlight area masks 334 may be generated torecover extra details in some highlight areas, for example in responseto the second input image data being mastered for an extended dynamicrange. The local high saturation area masks 336 may be used to markareas where extra colors are available in the second input image databecause of the use of a wider color gamut. Although three examples oftypes of masks are shown in FIG. 3 , other types of local masks can alsoor alternatively be generated depending on enhancement objectives. Inaddition, any type of local mask may be further categorized into moresub-groups of local masks based on image content, and each sub-group oflocal masks can be used for the transfer of a specific type of localattributes.

Returning to FIG. 2 , the content comparison processor 210 can generatea set of global feature transforms and a number of local mask sequencesfor enhancing the first format of display image data 20 via the processin FIG. 3 or otherwise. The output data 212 from the content comparisonprocessor 110 may be used directly by the display content generator 130to perform the image enhancement. In certain applications, however, theoutput data 212 may be stored for later processing, or may be sent to adifferent location to complete the enhancement. In those cases, theoutput data 212 can be packaged into a more compact form to facilitatestorage and transmission with higher efficiency and security. As shownin FIG. 2 , a metadata generator 220 can organize the output data 212from the content comparison processor 210 into a data file format thatis metadata 222.

The metadata 222 may have a container format and may serve as a wrapperof dynamic image and data information. The metadata 222 may have aflexible data structure that includes a header, a footer, and essencecontainers that can be frame-wrapped, clip-wrapped, segment-wrapped, orcustom-wrapped. The metadata 222 can include timecode information tosupport global feature transforms that may vary from image segment toimage segment, and to support local masks that may vary from frame toframe. The metadata 222 can also include global feature transforms anddifferent types of local masks, each a grayscale image sequence. Themetadata generator 220 may perform image coding so that the local masksequences or other frame-based information may be encoded in a highlycompressed form. The metadata generator 220 may also perform dataencryption to secure the encoded information. The metadata 222 may havea proprietary file format, and it may also use an industrialstandardized open file format such as MXF. The metadata 222 may be usedby the display content generator 230 to produce enhanced image data.

FIG. 4 is a flow chart of a process for producing enhanced image data bya display content generator according to one example of the presentdisclosure. In block 412, the metadata 222 is received by a metadatareceiver for use in modifying the first format of image data 20 usingthe information from the metadata 222. The format of the metadata 222can be determined to unwrap the global feature transforms for each imagesegment and to unwrap, and potentially decode, the local masks for eachframe.

In block 414, the received and decoded metadata may be analyzed, and theglobal feature transforms and local masks may be modified based on thirddisplay specifications 60, if such specifications 60 are received. Thethird display specifications 60, which may include data about thecharacteristics of the third display platform, may be optionallyreceived—e.g., in some cases the specifications 60 are not received. Ifthe specifications 60 are received, block 414 can account for thespecific characteristics of the third display platform and adjust theglobal feature transforms and local masks.

For example, a global feature transform may be modified based on certainrelevant information described in the third display specifications 60.Examples of the types of modifications applied to a global featuretransform can include scaling, interpolation, extrapolation, expansion,decomposition, linear, and non-linear mapping. In one example, a globalfeature transform is an electro-optical transfer function that may bescaled and interpolated, using the third display specifications, tomatch the dynamic range of the third display platform. In anotherexample, a global feature transform includes a color conversion matrix,and a linear mapping may be applied to match the color primaries andcolor gamut of the third display platform as reflected in the thirddisplay specifications 60. Examples of global feature transforms mayinclude color space conversion, dynamic range conversion, pixelresolution scaling, and luminance curve mapping.

The local masks decoded from the metadata 222 may also be analyzed andmodified based on the characteristics of the third display platform. Inone example, local highlight area masks can be analyzed and organizedinto a number of categories, including sparse highlight masks and spothighlight masks. Sparse highlight masks can include local highlightareas that are sparsely distributed over image frames. Examples of localhighlight areas that are sparsely distributed over image frames caninclude bright stars in a night sky or city lights over a darkbackground. Sparse highlight masks may be used to improve picturedynamic range for a third display platform, such as a direct view LEDdisplay or an OLED display, which provides individual pixel controls.Spot highlight masks include one or a number of local highlight areasthat are relatively large in size, or a cluster of highlight areas thatform a distinctive highlight region. Spot highlight masks may be used tooptimize picture dynamic range and overall picture quality for a thirddisplay platform, such as a projection system based on light steeringtechnology, that can control picture dynamic range in a selected region.In another example, local masks may be organized based on the artisticdirections of a content creator. The artistic direction of the contentcreator can be received through user input 430, which may include auser-friendly control device such as a graphic user interface, or even apre-recorded data format. From block 414, the modified global featuretransforms 416 and modified local masks 418 can be used to enhance thefirst format of display image data 20.

For example, in block 420, the modified global feature transforms 416are applied to modify the first format of display image data 20 toproduce enhanced first format of display image data. In one example,when one of the global feature transforms is HDR up-conversion, thetransforms can be applied to the first display image data 20 to increasepixel dynamic range. In another example, when color space conversion isone of the global feature transforms, the first display image data 20may be converted to a new color space with a wider color gamut.

In block 422, a layer decomposition process is applied to the enhancedfirst format display of image data to create a base layer and at leastone detail layer. A number of multi-scale image decomposition methodsmay be used to separate details into a single high resolution layer ormultiple higher resolution layers. Examples of image decompositionmethods can include pyramid methods, wavelet methods, and scale-spacemethods. Selecting a proper method can depend on the types of localattributes to be transferred from the second format of image data. Inone example, a wavelet decomposition method may be selected when extraimage details are to be transferred from the second format of imagedata.

In block 424, the base layer and the detail layers are modified usingmodified local masks 418 at pixel locations to create a new base layerand new detail layers. Extra image information carried over by the localmasks may be transferred to the base layer and the detail layers. Localoperations and transfers can be performed at the base layer and detaillayers based on the parameters provided in the local masks. As a result,the decomposed base layer and detail layers can be updated to form a newbase layer and new detail layers. In block 426, new image data,including new images, is created by reconstructing the new base layerand the new detailed base layer. The result can be outputted as enhancedimage data for display 70, which is an enhanced version of the firstformat of display image data 20.

Various types of systems can be used to implement various examples of animage enhancement process, including such a process that includes usingthe content comparison processor 210, the metadata generator 220, andthe display content generator 230 from FIG. 2 . Examples of the types ofsystems that can be used include a system with a computing device and asystem that includes multiple computing devices, each with certainresponsibilities and functions. FIG. 5 is a block diagram of a systemfor enhancing image data according to various examples, including theexamples described with reference to FIGS. 1 to 4 .

The system in FIG. 5 includes a processing device 502, a display device516, and a user interface 518. The processing device 502 includes aprocessor 512, a memory 504, a bus 510, and an input/output (I/O)interface 514. The processor 512 can execute one or more operations forgenerating enhanced image display for display on a third type of displaydevice using image data mastered for display on a first type of displaydevice and using image data mastered for display on a second type ofdisplay device. The processor 512 can execute instructions stored in thememory 504 to perform the operations. The processor 512 can include oneprocessing device or multiple processing devices or cores. Non-limitingexamples of the processor 512 include a Field-Programmable Gate Array(“FPGA”), an application-specific integrated circuit (“ASIC”), amicroprocessor, etc.

The processor 512 can be communicatively coupled to the memory 504 viathe bus 510. The non-volatile memory 504 may include any type of memorydevice that retains stored information when powered off. Non-limitingexamples of the memory 504 include EEPROM, flash memory, or any othertype of non-volatile memory. In some examples, at least part of thememory 504 can include a medium from which the processor 512 can readinstructions. A computer-readable medium can include electronic,optical, magnetic, or other storage devices capable of providing theprocessor 512 with computer-readable instructions or other program code.Non-limiting examples of a computer-readable medium include (but are notlimited to) magnetic disk(s), memory chip(s), ROM, RAM, an ASIC, aconfigured processor, optical storage, or any other medium from which acomputer processor can read instructions. The instructions can includeprocessor-specific instructions generated by a compiler or aninterpreter from code written in any suitable computer-programminglanguage, including, for example, C, C++, C #, etc.

The memory 504 can processor-executable instructions as an imageprocessing engine 506, along with a datastore 508. For example, theimage processing engine 506 can be executed by the processor 512 toperform the functions of the content comparison processor 210, themetadata generator 220, and the display content generator 230 from FIG.2 .

The I/O interface 514 can communicate with other components, for exampleto receive first format of display image data, second format of displayimage data, and third display specifications, which may be received viathe user interface 518. The I/O interface 514 can also output enhancedimage data, and other information for display by display device 516.

Other system configurations are possible. FIG. 6 is a block diagram of aconfiguration 602 for enhancing image data for display on a differenttype of display device according to one example of the presentdisclosure. In the configuration 602, the global feature transforms andlocal masks computed by the content comparison processor 210 can be useddirectly by the display content generator 230 without involving metadataor metadata generation. This configuration 602 can be applied when theprocessing workflow is executed at a single location. For example, theconfiguration 602 may be implemented by a single processing device toperform the functions of both the content comparison processor 210 andthe display content generator 230. That is, the first format of displayimage data 20 and the second format of display image data 30 is receivedand processed by the content comparison processor 210 (which may be partof the code stored on a computing device). The output of the contentcomparison processor 210 can be processed by the display contentgenerator 230 (which may be another part of the code stored on thecomputing device), along with accounting for any available third displayspecifications 60, to output enhanced image data for display 70.

FIG. 7 is a block diagram of a second configuration 704 for enhancingimage data for display on a different type of display device accordingto one example of the present disclosure. In the second configuration704, the functions of the content comparison processor 210 and themetadata generator 220 may performed at a first location (at which firstformat of display image data 20 and second format of display image data30 is received), and the functions of the display content generator 230are executed at a second location that can be distant from the firstlocation. The operations at the first location can include generatingthe metadata 222 in which the global feature transforms and local masksare encoded and packaged in a compact and secure form. The resultingmetadata 222 can be transmitted to the second location through networkor through a cloud-based platform 710. The metadata 222 can be receivedat the second location, along with the first format of display imagedata 20. A display content generator 230 can use the metadata 222, alongwith third display specifications (if available), to convert the firstformat of display image data 20 into enhanced image data for display 70.Transmitting metadata may be more secure than transmitting finalenhanced image data, and such a configuration may be helpful when theparty at the first location is a service provider and the party at thesecond location is a client with fewer computing resources and concernsfor data security.

In some aspects, methods and systems for enhancing image data areprovided according to one or more of the following examples:

Example #1: A method can include: receiving a first format of displayimage data mastered to be displayed on a first type of display system;receiving a second format of display image data mastered to be displayedon a second type of display system that is different than the first typeof display system; detecting one or more corresponding pixels in imagesegments among the first format of display image data and the secondformat of display image data; and using attributes from the one or morecorresponding pixels to generate enhanced image data for display.

Example #2: The method of Example #2 may feature using attributes fromthe one or more corresponding pixels to generate enhanced image data fordisplay by: generating segmented image data by comparing the firstformat of display image data and the second format of display image datato determine a plurality of image data similarities and a plurality ofimage data differences; determining, based on spatial correspondence ortemporal correspondence, a corresponding pixel pair between the firstformat of display image data and the second format of display image datafrom the segmented image data; defining a local mask sequence using thecorresponding pixel pair; and outputting the local mask sequence as amask file for generating enhanced image data to be displayed.

Example #3: The method of any of Examples #1-2 may feature the localmask sequence including at least one of a local dark area mask, a localhighlight mask, or a local high saturation area mask.

Example #4: The method of any of Examples #1-3 may feature the generatedenhanced image data being displayed on the first type of display system,the second type of display system, or a third type of display system.

Example #5: A method of image enhancement can include: receiving a firstformat of display image data mastered to be displayed on a first type ofdisplay system; receiving a second format of display image data masteredto be displayed on a second type of display system that is differentthan the first type of display system; generating segmented image databy comparing the first format of display image data and the secondformat of display image data to determine a plurality of image datasimilarities and a plurality of image data differences; determining,based on spatial correspondence or temporal correspondence, acorresponding pixel pair between the first format of display image dataand the second format of display image data from the segmented imagedata; defining a local mask sequence using the corresponding pixel pair;and outputting the local mask sequence as a mask file for generatingenhanced image data to be displayed.

Example #6: The method of Example #5 may feature the local mask sequenceincluding at least one of a local dark area mask, a local highlight areamask, or a local high saturation area mask.

Example #7: The method of any of Examples #5-6 can include: determining,based on a global feature, a related pixel group between the firstformat of display image data and the second format of display image datafor the segmented image data based on a global feature; determining aglobal feature transform by analyzing the related pixel group with theglobal feature; and outputting the global feature transform.

Example #8: The method of any of Examples #5-7 can include generating ametadata file based on the mask file and the global feature transform,the metadata file formatted to generate the enhanced image data.

Example #9: The method of any of Examples #5-8 can include: applying theglobal feature transform to modify the first format of display imagedata to produce enhanced first format of display image data; creating abase layer and a detailed layer by applying layer decomposition to theenhanced first format of display image data; modifying the base layerand the detailed layer using at least one mask to create a new baselayer and a new detailed layer; and reconstructing the new base layerand the new detailed layer to create new enhanced image data fordisplay.

Example #10: The method of any of Examples #5-9 may feature the firstformat of display image data being cinema image data.

Example #11: The method of any of Examples #5-10 can include: modifyinga parameter in the global feature transform based on a third type ofdisplay system having a third display specification with a third displayformat to create transformation information; modifying the first formatof display image data using the transformation information to create anenhanced first format image display data; decomposing the enhanced firstformat image display data to create a base layer and a detailed layer;modifying the base layer and the detailed layer with at least one maskto produce a new base layer and a new detailed layer; and reconstructingthe new base layer and the new detailed layer to create new enhancedimage data for display.

Example #12: The method of any of Examples #5-11 may feature the thirdtype of display system as a large format display and the first type ofdisplay system as a small format display.

Example #13: The method of any of Examples #5-12 may feature the thirdtype of display system being configured to display different brightnessimages than the first type of display system.

Example #14: The method of any of Examples #5-13 may feature the thirdtype of display system being configured to display a different field ofview than the first type of display system.

Example #15: The method of any of Examples #5-14 can include: analyzingand modifying a local highlight area mask based on a third displaycharacteristics, the local highlight area mask as modified is furthercategorized as a sparse local highlight mask or a spot local highlightmask; modifying a parameter in the global feature transform based on athird display specification with a third display format and creatingtransformation information based on the parameter as modified; modifyingthe first format of display image data using the transformationinformation, the sparse local highlight mask or the spot local highlightmask to create an enhanced first format image display data; decomposingthe enhanced first format data image data to create a base layer and adetailed layer; modifying the base layer and the detailed layer with atleast one mask to produce a new base layer and a new detailed layer; andreconstructing the new base layer and the new detailed layer to createnew enhanced image for display.

Example #16: A system can include a processing device and anon-transitory computer-readable memory comprising instructions that areexecutable by the processing device for causing the processing deviceto: receive a first format of display image data mastered to bedisplayed on a first type of display system; receive a second format ofdisplay image data mastered to be displayed on a second type of displaysystem that is different than the first type of display system; detectone or more corresponding pixels in image segments among the firstformat of display image data and the second format of display imagedata; and use attributes from the one or more corresponding pixels togenerate enhanced image data for display.

Example #17: The system of Example #16 may feature the memory furthercomprising instructions that are executable by the processor for causingthe processor to use attributes from the one or more correspondingpixels to generate enhanced image data for display by: generatingsegmented image data by comparing the first format of display image dataand the second format of display image data to determine a plurality ofimage data similarities and a plurality of image data differences;determining, based on spatial correspondence or temporal correspondence,a corresponding pixel pair between the first format of display imagedata and the second format of display image data from the segmentedimage data; defining a local mask sequence using the corresponding pixelpair; and outputting the local mask sequence as a mask file forgenerating enhanced image data to be displayed.

Example #18: The system of any of Examples #16-17 may feature the localmask sequence including at least one of a local dark area mask, a localhighlight mask, or a local high saturation area mask.

Example #19: The system of any of Examples #16-18 may feature thegenerated enhanced image data being displayed on the first type ofdisplay system, the second type of display system, or a third type ofdisplay system.

Example #20: An image enhancement system may include a processingdevice; and a non-transitory computer-readable memory comprisinginstructions that are executable by the processing device for causingthe processing device to: receive a first format of display image datamastered to be displayed on a first type of display system; receive asecond format of display image data mastered to be displayed on a secondtype of display system that is different than the first type of displaysystem; generate segmented image data by comparing the first format ofdisplay image data and the second format of display image data todetermine a plurality of image data similarities and a plurality ofimage data differences; determine, based on spatial correspondence ortemporal correspondence, a corresponding pixel pair between the firstformat of display image data and the second format of display image datafrom the segmented image data; define a local mask sequence using thecorresponding pixel pair; and output the local mask sequence as a maskfile for generating enhanced image data to be displayed.

Example #21: The image enhancement system of Example #20 may feature thelocal mask sequence including at least one of a local dark area mask, alocal highlight area mask, or a local high saturation area mask.

Example #22: The image enhancement system of any of Examples #20-21 mayfeature the memory further comprising instructions that are executableby the processing device for causing the processing device to:determine, based on a global feature, a related pixel group between thefirst format of display image data and the second format of displayimage data for the segmented image data based on a global feature;determine a global feature transform by analyzing the related pixelgroup with the global feature; and output the global feature transform.

Example #23: The image enhancement system of any of Examples #20-22 mayfeature the memory further comprising instructions that are executableby the processing device for causing the processing device to: generatea metadata file based on the mask file and the global feature transform,the metadata file formatted to generate the enhanced image data.

Example #24: The image enhancement system of any of Examples #20-23 mayfeature the memory further comprising instructions that are executableby the processing device for causing the processing device to: apply theglobal feature transform to modify the first format of display imagedata to produce enhanced first format of display image data; create abase layer and a detailed layer by applying layer decomposition to theenhanced first format of display image data; modify the base layer andthe detailed layer using at least one mask to create a new base layerand a new detailed layer; and reconstruct the new base layer and the newdetailed layer to create new enhanced image data for display.

Example #25: The image enhancement system of any of Examples #20-24 mayfeature the first format of display image data as cinema image data.

Example #26: The image enhancement system of any of Examples #20-25 mayfeature the memory further comprising instructions that are executableby the processing device for causing the processing device to: modify aparameter in the global feature transform based on a third type ofdisplay system having a third display specification with a third displayformat to create transformation information; modify the first format ofdisplay image data using the transformation information to create anenhanced first format image display data; decompose the enhanced firstformat image display data to create a base layer and a detailed layer;modify the base layer and the detailed layer with at least one mask toproduce a new base layer and a new detailed layer; and reconstruct thenew base layer and the new detailed layer to create new enhanced imagedata for display.

Example #27: The image enhancement system of any of Examples #20-26 mayfeature the third type of display system being a large format displayand the first type of display system being a small format display.

Example #28: The image enhancement system of any of Examples #20-27 mayfeature the third type of display system being configured to displaydifferent brightness images than the first type of display system.

Example #29: The image enhancement system of any of Examples #20-28 mayfeature the third type of display system being configured to display adifferent field of view than the first type of display system.

Example #30: The image enhancement system of any of Examples #20-29 mayfeature the memory further comprising instructions that are executableby the processing device for causing the processing device to: analyzeand modify a local highlight area mask based on a third displaycharacteristics, the local highlight area mask as modified is furthercategorized as a sparse local highlight mask or a spot local highlightmask; modify a parameter in the global feature transform based on athird display specification with a third display format and creatingtransformation information based on the parameter as modified; modifythe first format of display image data using the transformationinformation, the sparse local highlight mask or the spot local highlightmask to create an enhanced first format image display data; decomposethe enhanced first format data image data to create a base layer and adetailed layer; modify the base layer and the detailed layer with atleast one mask to produce a new base layer and a new detailed layer; andreconstruct the new base layer and the new detailed layer to create newenhanced image data for display.

The foregoing is provided for purpose of illustrating, explaining, anddescribing embodiments of the present invention. Further modificationsand adaptations to those embodiments will be apparent to those skilledin the art and may be made without departing from the scope or thespirit of the invention.

What is claimed is:
 1. A method comprising: receiving a first format ofdisplay image data mastered to be displayed on a first type of displaysystem; receiving a second format of display image data mastered to bedisplayed on a second type of display system that is different than thefirst type of display system; detecting one or more corresponding pixelsin image segments among the first format of display image data and thesecond format of display image data; and using attributes from the oneor more corresponding pixels to generate enhanced image data fordisplay.
 2. The method of claim 1, wherein using attributes from the oneor more corresponding pixels to generate enhanced image data for displayfurther comprises: generating segmented image data by comparing thefirst format of display image data and the second format of displayimage data to determine a plurality of image data similarities and aplurality of image data differences; determining, based on spatialcorrespondence or temporal correspondence, a corresponding pixel pairbetween the first format of display image data and the second format ofdisplay image data from the segmented image data; defining a local masksequence using the corresponding pixel pair; and outputting the localmask sequence as a mask file for generating enhanced image data to bedisplayed.
 3. The method of claim 2, wherein the local mask sequenceincludes at least one of a local dark area mask, a local highlight areamask, or a local high saturation area mask.
 4. The method of claim 1,wherein the generated enhanced image data is displayed on the first typeof display system, the second type of display system, or a third type ofdisplay system.
 5. A method of image enhancement comprising: receiving afirst format of display image data mastered to be displayed on a firsttype of display system; receiving a second format of display image datamastered to be displayed on a second type of display system that isdifferent than the first type of display system; generating segmentedimage data by comparing the first format of display image data and thesecond format of display image data to determine a plurality of imagedata similarities and a plurality of image data differences;determining, based on spatial correspondence or temporal correspondence,a corresponding pixel pair between the first format of display imagedata and the second format of display image data from the segmentedimage data; defining a local mask sequence using the corresponding pixelpair; and outputting the local mask sequence as a mask file forgenerating enhanced image data to be displayed.
 6. The method of claim5, wherein the local mask sequence includes at least one of a local darkarea mask, a local highlight area mask, or a local high saturation areamask.
 7. The method of claim 6, further comprising: determining, basedon a global feature, a related pixel group between the first format ofdisplay image data and the second format of display image data for thesegmented image data based on a global feature; determining a globalfeature transform by analyzing the related pixel group with the globalfeature; and outputting the global feature transform.
 8. The method ofclaim 7, further comprising: generating a metadata file based on themask file and the global feature transform, the metadata file formattedto generate the enhanced image data.
 9. The method of claim 7, furthercomprising: applying the global feature transform to modify the firstformat of display image data to produce enhanced first format of displayimage data; creating a base layer and a detailed layer by applying layerdecomposition to the enhanced first format of display image data;modifying the base layer and the detailed layer using at least one maskto create a new base layer and a new detailed layer; and reconstructingthe new base layer and the new detailed layer to create new enhancedimage data for display.
 10. The method of claim 5, wherein the firstformat of display image data is cinema image data.
 11. The method ofclaim 7, further comprising: modifying a parameter in the global featuretransform based on a third type of display system having a third displayspecification with a third display format to create transformationinformation; modifying the first format of display image data using thetransformation information to create an enhanced first format imagedisplay data; decomposing the enhanced first format image display datato create a base layer and a detailed layer; modifying the base layerand the detailed layer with at least one mask to produce a new baselayer and a new detailed layer; and reconstructing the new base layerand the new detailed layer to create new enhanced image data fordisplay.
 12. The method of claim 11, wherein the third type of displaysystem is a large format display and the first type of display system isa small format display.
 13. The method of claim 11, wherein the thirdtype of display system is configured to display different brightnessimages than the first type of display system.
 14. The method of claim11, wherein the third type of display system is configured to display adifferent field of view than the first type of display system.
 15. Themethod of claim 7, further comprising: analyzing and modifying a localhighlight area mask based on a third display characteristics, the localhighlight area mask as modified is further categorized as a sparse localhighlight mask or a spot local highlight mask; modifying a parameter inthe global feature transform based on a third display specification witha third display format and creating transformation information based onthe parameter as modified; modifying the first format of display imagedata using the transformation information, the sparse local highlightmask or the spot local highlight mask to create an enhanced first formatimage display data; decomposing the enhanced first format data imagedata to create a base layer and a detailed layer; modifying the baselayer and the detailed layer with at least one mask to produce a newbase layer and a new detailed layer; and reconstructing the new baselayer and the new detailed layer to create new enhanced image fordisplay.
 16. A system comprising: a processing device; and anon-transitory computer-readable memory comprising instructions that areexecutable by the processing device for causing the processing deviceto: receive a first format of display image data mastered to bedisplayed on a first type of display system; receive a second format ofdisplay image data mastered to be displayed on a second type of displaysystem that is different than the first type of display system; detectone or more corresponding pixels in image segments among the firstformat of display image data and the second format of display imagedata; and use attributes from the one or more corresponding pixels togenerate enhanced image data for display.
 17. The system of claim 16,wherein the memory further comprises instructions that are executable bythe processor for causing the processor to use attributes from the oneor more corresponding pixels to generate enhanced image data for displayby: generating segmented image data by comparing the first format ofdisplay image data and the second format of display image data todetermine a plurality of image data similarities and a plurality ofimage data differences; determining, based on spatial correspondence ortemporal correspondence, a corresponding pixel pair between the firstformat of display image data and the second format of display image datafrom the segmented image data; defining a local mask sequence using thecorresponding pixel pair; and outputting the local mask sequence as amask file for generating enhanced image data to be displayed.
 18. Thesystem of claim 17, wherein the local mask sequence includes at leastone of a local dark area mask, a local highlight area mask, or a localhigh saturation area mask.
 19. The system of claim 16, wherein thegenerated enhanced image data is displayed on the first type of displaysystem, the second type of display system, or a third type of displaysystem.
 20. An image enhancement system comprising: a processing device;and a non-transitory computer-readable memory comprising instructionsthat are executable by the processing device for causing the processingdevice to: receive a first format of display image data mastered to bedisplayed on a first type of display system; receive a second format ofdisplay image data mastered to be displayed on a second type of displaysystem that is different than the first type of display system; generatesegmented image data by comparing the first format of display image dataand the second format of display image data to determine a plurality ofimage data similarities and a plurality of image data differences;determine, based on spatial correspondence or temporal correspondence, acorresponding pixel pair between the first format of display image dataand the second format of display image data from the segmented imagedata; define a local mask sequence using the corresponding pixel pair;and output the local mask sequence as a mask file for generatingenhanced image data to be displayed.
 21. The image enhancement system ofclaim 20, wherein the local mask sequence includes at least one of alocal dark area mask, a local highlight area mask, or a local highsaturation area mask.
 22. The image enhancement system of claim 21,wherein the memory further comprises instructions that are executable bythe processing device for causing the processing device to: determine,based on a global feature, a related pixel group between the firstformat of display image data and the second format of display image datafor the segmented image data based on a global feature; determine aglobal feature transform by analyzing the related pixel group with theglobal feature; and output the global feature transform.
 23. The imageenhancement system of claim 22, wherein the memory further comprisesinstructions that are executable by the processing device for causingthe processing device to: generate a metadata file based on the maskfile and the global feature transform, the metadata file formatted togenerate the enhanced image data.
 24. The image enhancement system ofclaim 22, wherein the memory further comprises instructions that areexecutable by the processing device for causing the processing deviceto: apply the global feature transform to modify the first format ofdisplay image data to produce enhanced first format of display imagedata; create a base layer and a detailed layer by applying layerdecomposition to the enhanced first format of display image data; modifythe base layer and the detailed layer using at least one mask to createa new base layer and a new detailed layer; and reconstruct the new baselayer and the new detailed layer to create new enhanced image data fordisplay.
 25. The image enhancement system of claim 20, wherein the firstformat of display image data is cinema image data.
 26. The imageenhancement system of claim 22, wherein the memory further comprisesinstructions that are executable by the processing device for causingthe processing device to: modify a parameter in the global featuretransform based on a third type of display system having a third displayspecification with a third display format to create transformationinformation; modify the first format of display image data using thetransformation information to create an enhanced first format imagedisplay data; decompose the enhanced first format image display data tocreate a base layer and a detailed layer; modify the base layer and thedetailed layer with at least one mask to produce a new base layer and anew detailed layer; and reconstruct the new base layer and the newdetailed layer to create new enhanced image data for display.
 27. Theimage enhancement system of claim 26, wherein the third type of displaysystem is a large format display and the first type of display system isa small format display.
 28. The image enhancement system of claim 26,wherein the third type of display system is configured to displaydifferent brightness images than the first type of display system. 29.The image enhancement system of claim 26, wherein the third type ofdisplay system is configured to display a different field of view thanthe first type of display system.
 30. The image enhancement system ofclaim 22, wherein the memory further comprises instructions that areexecutable by the processing device for causing the processing deviceto: analyze and modify a local highlight area mask based on a thirddisplay characteristics, the local highlight area mask as modified isfurther categorized as a sparse local highlight mask or a spot localhighlight mask; modify a parameter in the global feature transform basedon a third display specification with a third display format andcreating transformation information based on the parameter as modified;modify the first format of display image data using the transformationinformation, the sparse local highlight mask or the spot local highlightmask to create an enhanced first format image display data; decomposethe enhanced first format data image data to create a base layer and adetailed layer; modify the base layer and the detailed layer with atleast one mask to produce a new base layer and a new detailed layer; andreconstruct the new base layer and the new detailed layer to create newenhanced image data for display.